Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity elds for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using blockor object-based methods. A dynamic programming algorithm is used to estimate a disparity eld between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity elds. Spatial interpolation techniques are examined based on the disparity=depth information and the detection of occluded regions using either stereoscopic or trinocular camera con gurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity eld and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion elds needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods. c © 1998 Elsevier Science B.V.